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SUMMARY 


Unless  techniques  are  developed  to  solve  special 
classes  ol'  linear  programming  problems,  it  is  likely  that 
only  models  Involving  between  100  and  200  equations  Vi’lll 
be  successfully  computed  by  general  techniques  now'  avail¬ 
able.  This  paper  discusses  tlie  need  to  compute  large 
scale  si's'tems  and  points  out  some  of  the  common  character¬ 
istics  of  many  practical  models  which  promise  to  lead  to 
short  cut  procedures. 
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STATUS  OF  SOLUTION  OF  LARGE-SCALE 
LINEAR  PROGRAMMING  PROBLEMS 

Talk  be?fore 

Institute  of  Management  Sciences 
Pittsburgh j  Pennsylvania 
October  22,  195^ 


by 

George  B,  Dantzig 

The  purpose  of  my  talk  is  to  discuss  with  you  the  possibilities 
for  solving  large  scale  linear  programming  problems.  I  shall  soiind 
both  an  optimistic  and  a  pessimistic  note.  The  pessimistic  note 
concerns  the  ability  of  the  problem  formulator,  either  amateur  or 
professional,  to  develop  models  that  are  large  scale.  The  pessi¬ 
mistic  note  also  concerns  the  Inability  of  the  problem  solver  to 
com.pute  models  by  general  techniques  v/hen  they  are  large  scale.  If 
this  is  so,  is  not  the  great  promise  that  the  linear  programming 
approach  will  solve  scheduling  and  long  range  planning  problems  ^^rith 
substantial  savings  to  the  organizations  adopting  these  methods  but 
an  illusion  and  a  snare?  Are  the  big  problems  going  to  be  solved  as 
they  have  always  been  solved  —  by  a  detailed  system  of  on— the— spot 
somewhat  natural  set  of  priorities  that  resolve  every  possible 
alternative  as  it  arises? 

Let  us  consider  a  modest  planner  who  is  concerned  with  the 
expansion  of  motor  production  —  let  us  say  a  special  type  motor 
that  requires  a  special  type  of  steel  and  must  use  tools  fabricated 
from  this  steel  and  the  tools  which  fabricate  these  tools  also  use 
this  steel.  The  tools  that  fabricate  steel  v/e  will  call  steel 
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capacity,  those  that  fabricate  tools  —  tool  capacity,  and  tnose 
that  fabricate  motors  —  motor  capacity.  Tne  planner  is  quite  modest 
because  he  is  willing  to  consolidate  all  of  the  multitudinous 
activities  and  items  into  these  simple  terms.  He  Is  interested  in 
developing  a  program  over  two  years  by  quarters,  that  meets  a  speci¬ 
fied  schedule  of  knov/n  sales  and  creates  the  largest  stockpile  of 
motors  for  any  future  sales  that  may  develop.  The  initial  Inventory 
must  satisfy  the  following  relations  in  detached  coefficient  form! 
for  first  tvro  time  periods; 
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The  pattern  must  be  repeated  for  eight  time  periods.  If  v/e 
denote  the  upper  and  lower  blocks  by  A  and  B  respectively,  the  model 
has  the  form 


10  X  8 


The  resulting  system  of  40  equations  in  80  variables  v/lth  the 
objective  to  maximize  a  stockpile  of  motors  can  be  solved  in  a  half 
hour  on  a  modern  electronic  computer.  Let  this  planner  now  decide 
that  his  model  is  entirely  too  coarse  and  that  he  must  plan  by 
months ,  distinguish  two  types  of  motors  and  tvjo  types  of  steel  and 
our  resultant  system  becomes  7  x  24,  14  x  24  or  lb4  x  836 .  At 

this  size  the  computation  would  require  about  one  v/eek  using  one  shift 
per  day.  From  the  viewpoint  of  the  computer  the  planner  is  no  longer 
modest.  However,  for  those  of  you  in  the  audience  who  have  ever 
engaged  in  programming,  it  Is  clear  that  the  so-called  "detailed" 
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model  above  is  at  best  only  useful  as  an  over— all  type  of  guide, 
but  hardly  detailed  in  a  realistic  sense. 

Let  me  cite  an  example  from  another  area  —  the  problem  of 
routing  cargo  aircraft.  Let  the  variable  represent  the 

number  of  aircraft  of  type  k  routed  between  city  1  and 
l.ct  us  distinguish  between  six  types  of  aircraft,  ten  time  periods, 
and  tiventy  cities.  In  addition,  consider  a  second  set  of  variables 
which  is  the  tons  of  cargo  shipped  between  city  1  and  j  ^ 


on  the  way  to  i.  Our  equations  become 


V  ^  / 
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Tonage  Cap.  >  Tonage  Req.: 
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Piarie  Moriths  Availaole:  Y  T  a.  x,,  =  p,. 
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As  we  see  again  such  a  system  involving  only  a  fevf  cities,  type 
aircraft,  and  cargo  destinations  generate  easily  a  system  in  a  1000 
equations  in  10,000  unknowns.  Superficially,  a  very  discouraging 


situation . 


RM-13T5 

11-50-54 

-5~ 


For  a  number  of  years  research  has  been  going  on,  now  centered 
at  RAND,  specifically  directed  toward  the  solution  of  large  scale 
systems  and  my  talk  to  you  today  is  in  a  nature  of  a  progress  report. 
At  HAND  v;e  have  a  simplex  code  which  can  successfully  solve  a  system 
of  100  equations  in  any  number  of  unknowns  in  about  five  hours.  By 
successful  I  mean  it  has  done  so  on  a  number  of  occasions  and  has 
oroduced  accurate  usable  results. 


Theoretically  the  machine  could  do  200  equations  but  It  would 
take  so  long  that,  with  machine  failures,  restarts,  etc.,  it  does 
not  appear  practical  to  do  so.  We  also  have  a  special  code  for  a 
special  type  of  problem  called  the  "machine  processing"  model. 


Matriemsttlcally,  this  may  be  described  as  a  slightly 
of  transportation  model.  A  transportation  model  as 
Hitchcock  and  later  independently  by  T.  C.  Koopirans 


generalized  type 
developed  by 
is  of  the  form 
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We  can  sligiitly  generalize  it  by  replacing  the  coefficient  of 

X.  .  in  the  first  equation  by  A.  .. 

J  J 

The  special  RAND  code  which  only  works  for  a  particular  choice 

of  c  .  can  work  out  solutions  to  a  5U0— eqtiation  system  of  this 
1  J 
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\type  in  anout  an  hour. 

Basically,  our  approacii  has  oeen  to  study  a  large  number  of 
models  as  they  arise  in  practice  to  discover  if  they  have  any  char¬ 
acteristic  that  can  be  taken  advantage  of  computationally.  When  we 
have  developed  a  theoretical  approach  we  often  take  a  watered— do'wn 
version  of  the  model  that  can  be  solved  directly  by  the  general  code 
and  compare  this  with  the  new  method.  Often  these  ne’w  methods  are 
powerful  enough  to  do  by  hand  what  cannot  be  achieved  by  machine. 

Of  course,  situations  of  this  sort  are  not  new.  A  transportation 
problem  of  the  Hitchcock— Koopmans  variety  involving,  say,  a  hundred 
rov/s  and  columns  combined  or  one  involving  10  rows  and  hundreds  of 
columns  can  be  nicely  handled  by  clerks.  Recently,  Cooper  and 
Charnes  and  others  have  reported  that  they  were  si^ccessful  in  solving 
certain  very  large  systems  by  hand  methods.  All  of  this  gives  en¬ 
couragement  to  believe  that  many  large  scale  systems  will  be 
successfully  solved  in  the  future. 

Except  in  highly  specialized  models  of  the  transportation  type 
or  others  where  unusual  characteristics  can  be  t?iken  advantage  of  , 
greauer  progress  will  be  made  by  developing  in  t/ie  first  phase 
consolidated  versions  of  the  model.  There  are  several  reasons  for 
this.  In  the  first  place  this  effort  results  In  a  model  v/hich  is 
often  very  useful  in  Itself.  Secondly,  it  provides  an  excellent 
dry  run  for  methodology.  In  a  word.  It  is  better  for  administrative 
and  technical  reasons  to  keep  the  model  initially  small. 


Inevitably  the  size  of  models  does  increase.  I  wish  to  discuss 
some  devices  that  can  greatly  reduce  the  amount  of  computation.  In 
the  first  place  there  appears  to  be  a  number  of  important  character¬ 
istics  commonly  found  in  pt-actical  models. 

(a)  Matrix  of  coefficients  is  composed  of  blocks  of  zeros. 

(b)  Within  non— zero  blocks  most  elements  are  zero. 

(c)  Matrix  is  often  block  triangular. 

(d)  Many  variables  have  simple  upper  bounds  or  satisfy 

a  system  of  secondary  constraints,  most  of  wblcr:  are 
non— active  in  any  given  problem. 

Block  triangularity  is  one  of  the  most  promising  characteristics 


to  exploit.  The  basis  in  the  simplex  metiiod  consists  in  a  selected 
subset  of  the  columns  of  the  coefficients.  It  Is  square,  rjon— 
singular  matrix  which  from  one  cycle  to  tiie  next  is  modified  by  one 
column.  Its  Inverse  or  equivalent  is  needed  on  eacri  itera.tion.  If 
the  oasis  is  of  tnls  form  (  1  ),  and  let  us  suppose  tiie  diagonal 
submiatrlces  designated  A  (but  not  necessarily  the  same)  are  square 
and  non— singular.  It  it  has  this  property  it  v/ill  be  referred  to 
as  "square  block  triangular."  This  form  is  ideal  because  it  is 
necessary  only  to  have  the  Inverses  of  the  diago.sai  submaurices . 
Furthermore,  from  one  iteration  to  the  next  it  becomes  only  neces¬ 


sary  to  modify  one  of  these  smaller  inverses.  Strictly  speaking 
most  bases  taken  from  block  triangular  sysuems  do  not  coriform  to 
this  ideal  situation  but  are  very  close  to  ceing  of  the  required 
form.  In  this  case,  it  is  possible  by  transforming  tlie  matrix 
slightly  to  get  into  the  required  square  block  triangular  form. 


r 
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Vihen  a  matrix  Is  composed  largely  of  zeros  v/here  the  zeros  are 
in  no  obvious  pattern,  it  is  often  practical  to  solve  directly  for 
the  prices  and  the  representations  of  the  vectors  entering  the  basis 
rather  than  to  solve  for  them  bjr  means  of  the  inverses  of  siiccessive 
bases.  The  transportation  model  is  a  classical  case  where  this 
approach  has  paid  off. 

When  many  variables  have  simple  upper  bounds,  it  is  no  longer 
necessary  to  add  one  more  variable  and  equation  for  eacn  sucri 
restraint.  Instead,  it  is  possible  to  slightly  modify  the  original 
simplex  algorithm  arid  apply  It  to  the  system  excluding  toe  upper 
bounds.  Essentially,  one  replaces  a  variable  by  x  -  b  ~  x  where 
b  is  an  upper  bound  for  x  ivhenever  in  the  simplex  process  x  reaches 
its  upper  bound. 

In  many  problems  there  are  equations  that  may  be  considered 
as  forming  a  set  of  side  coriditions;  for  example,  conditions  that 
capacity  of  certain  machines  is  never  exceeded  or  the  character¬ 
istics  of  certain  products;  e.g.,  its  viscosity  is  within 
specifications,  etc.  In  most  problems  only  a  small  subset  of  these 
"secondary"  constraints  are  likely  to  oe  active,  l.e.,  at  their 
cr’ltical  value — the  others  being  ’well  within  specifications  or 
capacities.  In  suchi  eases  it  is  recommended  that  the  linear 
programraing  proolem  be  first  solved  without  regard  to  these  secondary 
constraints.  Then  the  system  is  enlarged  to  include  the  secondary 
constraints  and  an  Initial  basis  is  obtained  by  augmenting  the 
final  basis  of  the  smaller  Droblern.  This  vjill  result  in  a  basis 
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in  which  not  all  variables  associated  v^ith  the  secondary  constralriLi 
are  positive.  Hov'/ever,  in  this  form  the  dual  simplex  procedure  of 
Lemke  may  be  employed.  Often  in  practical  cases  only  a  fev.'  iter¬ 
ations  are  needed  to  clean  up  the  negative  variables  and  obtain  an 
optimum  solution . 

Details  of  the  three  procedures  outlined  above  may  be  found 
in  FiAND  "Notes  on  Linear  Programming:  Parts  VIII,  IX,  X  —  Upper 
Bounds,  Secondary^  Constraints,  arid  Block  Triangularity  in  .Linear 
Programming,"  dated  4  October  1954- 


